Generating Function Notes 
18.310, Fall 2005, Prof. Peter Shor 




1 Counting Change 

In this lecture, I'm going to talk about generating functions. We've already seen 
an example of generating functions. Recall when we had n independent random 
variables, each of which was with probability p, and 1 with probability q = 1 — p. 
How did we compute the probability of getting exactly k Vs. What we did was look 
at the polynomial 

{P + = (^^P"- + (^^P^'-'qx + (^^p--\'x^ + ...+ 

The coefficient of was the probability of getting k I's. This is a simple generating 
function, and we will generalize this construction in today's lecture. There are a lot 
more things you can do with generating functions that aren't going to be covered 
in today's lecture, but it will give you a flavor of how these work. It will also show 
you how to solve linear recurrence equations, which can be very useful. 

Let's look at a related question. Suppose we have 4 pennies, 1 nickel, and 1 
dime. For which prices can we give exact change? The answer can be found in 
a way that's a lot like the probability computation above (in fact, you may have 
figured out the answer already, but this is an easy example to show you how these 
generating functions work). 

We'll introduce a variable x so that re" will correspond to the ways of making n 
cents in change. We can either use 0, 1, 2, 3, or 4 pennies, and or 1 nickels, and 
or 1 dimes. Thus, 

(1 + X^^){1 + x^){l + x + x'^ + x^ + x^) 

is the generating function which gives the answer. If we use the dime, we use the 
x^^ monomial in the first term. Otherwise, we use the x^ = 1 monomial, and so 
forth. Multiplying this out, we get 

(1 + + + x + x"^ + x'^ + x"^) = 1 + x + x'^ + x^ + . . . + x^^, 

showing that we can obtain any number of cents between and 19. 

A little thought will convince you that the coefficient on the x'^ term gives the 
number of different ways of finding coins that add up to k cents. We can thus use 
the same technique to answer a question such as: How many ways can you make 
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21 cents exact change if you have two dimes, three nickels, and seven pennies? For 
this problem, the generating function is 



(1 + x^^ + + x^ + + x^^){l + x + x'^ +x^ + x'^ + x^ + x^ + x'^). 

The first term is the contribution from the dimes, the second term from the nickels, 
and the third from the pennies. The coefficient on the term gives you the 
number of way of making 21 cents exact change. This coefficient is 4, and the terms 
correspond to coins as follows 

l - X two dimes and one penny 

^10 . ^10 . ^ Qj^g dime, two nickels and one penny 

^10 . ■ one dime, one nickel and six pennies 

1 ■ x^^ ■ x^ three nickels and six pennies 

What happens if you have an arbitrary number of dimes, nickels, and pennies? 
Now, we want to multiply by (for the dimes) 



where we can have any non-negative integer for the number of dimes. But this 
infinite series above is just 1/(1 — x^^), and it converges as long as \x\ < 1. Thus, 
the generating function that corresponds to making change using any number of 
dimes, nickels, and pennies, would be 



(l-x)(l-x5)(l-a;i0)' 

The corresponding power series converges if |a;| < 1. The essential ingredient we 
need to know in terms of using it is that it converges in some neighborhood around 
the X = 0. Power series that don't necessarily converge anywhere are called formal 
power series; these can still be useful for some purposes, but we won't discuss them 
any further in this class. 

Let's do one more thing with change before we move on. Suppose we are paying 
with dollar bills and dollar coins. If we count these two forms of money as different, 
we can make n dollars in n + 1 different ways: we can use k coins and n — k bills 
for any value of k between and n. This corresponds to the identity 

^ 1 + 2x1°° + 3x200 + 4x300 + ... 



_ ^loo^2 



(1-x 
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2 Dots and Dashes 



Now, let's think about another problem. Suppose that you arc sending information 
using a sequence of two symbols, say dots and dashes, and suppose that the dash is 
two times as long as a dot^. How many different messages can you send in n time 
units? Let's call this number /„. We'll figure /„ out for the first few n. We have 
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You may already recognize the pattern: these are Fibonacci numbers. But let's see 
what we can find out about Fibonacci numbers using generating functions. The 
recursion for the Fibonacci numbers is 

fn = fn-l + fn-2 

It's not too difficult to see see why this works. The last symbol must be either a 
dot or a dash. If the last symbol is a dash, removing it leaves a sequence two units 
shorter. If the last symbol is a dot, removing it leaves a sequence one unit shorter. 
Now, how does this connect to generating functions? Let us define 

oo 

9{x) = ^fjx^- 

j=0 

What is /o? It has to be 1, in order to have /2 = /i+/o- This makes sense intuitively: 
there is one message, the empty message, using zero iniits of time (intuitively, setting 
/o = also makes sense, but it doesn't satisfy the recurrence). What does this 
recurrence say about g{x)7 Let's look at the following equations 

g(x) = 1 + X + 2x'^ + 3x^ + 5x^ + 8x^ + ISx^ 
xg{x) = X + x"^ + 2x^ + 2>x'^ + + Sx*^ 

x'^glx) = x'^ + x^ + 2x^ + 3x^ + 

Wc can see that by multiplying by x and we have shifted the terms so that 
instead of terms fkX^, we get terms fk-ix^ and fk-2X^- We thus get that the 

^Real Morse code is a bit more complicated. 
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equation fk = fk-i+fk-2 seems to correspond to the (slightly incorrect) equation 

g{x) = xg{x) + x'^g{x). Is this the right equation for our generating function g{x)7 
It isn't quite right. All the terms x'^ for A; > do cancel in this equation, but it 
doesn't work for the constant term. To make the constant term correct, we need to 
add 1 to the right side, obtaining the correct equation 

g{x) = xg{x) + x^g{x) + 1. 

Now, this can be rewritten as 

1 



9{x) 



1 — x — x^ 



When we have a polynomial in the denominator of a fraction like this, we can factor 
the polynomial and express it as the sum of two simpler fractions. That is, we first 
factor the denominator 



X — x^ 



We now use the method of partial fractions to rewrite this as 



9ix) = -T^Tf— + 
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Some of you may have seen the method of partial fractions before in calculus when 
you learned integration. We need to find a and h. Elementary algebra gives 

Now, we need to remember what the Taylor series for 1/(1 — ax) is. It is 
^ =l + ax + o?x^ + a^x^ + a^x^ + . . . 



OLX 



Even those of you who haven't seen Taylor expansions should recognize this as the 
formula for summing a geometric series. 
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We thus have that, expanding each of the fractions in the expression for g{x) 
above by a Taylor series, 

+b (l + (1^) + {^fx^ + (i^)%3 + . . 



^^/^ 11 o ;ii o y-^ii o I ^ \ \ o i x' ~\- . . . 

Now, this gives us a nice expression for /„, the nth Fibonacci number. We equate 
the coefficients of on the left- and right-hand sides of this equation. Since the 
nth Fibonacci number /„ is the coefficient on in g{x), we get 

Now, since < 1, we can see that the second term above is neglible, and /„ 

grows as 

C(i±^)" 

for some C. (And in fact, it's really easy to figure out C from this formula.) This 
means that we have found the asymptotic growth rate for the number of messages 
that can be encoded by our dots and dashes, and that number of bits sent per time 
unit is 

Now let's generalize this. Tm going to give you a general method for solving 
linear recurrence equations (also called linear difference equations). If you've taken 
18.03, you'll notice that this method looks a lot like the method for solving linear 
differential equations, and in general, it turns out that recurrence equations often 
behave a lot like differential equations. 

Suppose we have a recurrence equation 

fn = afn-l + Pfn-2 + lfn-3 

I'm only writing this equation down with three terms, but the generalization to A; 
terms is obvious, and works just like you'd expect. How do we solve this? What we 
do is we write down the generating function 

oo 
j=0 
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We then, using the same reasoning we had before, get an equation for g{x) of the 
fohowing form: 

g{x) = axg{x) + (3x'^g{x) + ^x^g{x) +p{x) 

where p{x) is a low-degree polynomial that makes this equation work for the first 
few elements of the sequence, where the recurrence equation doesn't work (because 
we don't have a /_i term). For the Fibonacci number example above, we have 
p{x) = 1. Note that if we don't have a p{x) term, we get the solution g{x) = 
which, while its coefficients (all O's) satisfy the linear recurrence equation, doesn't 
tell us much useful. 

We then obtain, as before 

p{x) 

9{x) = 



1 — ax — Px"^ — 'jx^ 
Let's suppose we can factor the denominator as follows: 

1 — ax — /3x^ — jx^ = (1 — rix){l — r2x)(l — rsx). 

We then use the method of partial fractions (which you may remember from Cal- 
culus) to get 

, , , . a b c 
g{x) = q{x) + + + 

1 — rix 1 — r2X 1 — r^x 

where a, b and c are constants, as before, and where q{x) is a low degree polynomial 
that takes care of any anomolies in the first few terms of the sequence (there usually 
won't be any, so q{x) usually will be 0). We can then see, by taking a Taylor 
expansion for this generating function, that a generic term of our sequence will be 

/„ = ar^ + br^ + cr^ 

How did we get the roots ri, r2 and r^? They are the zeros of the polynomial 

- ay^ - /3y - 7. 

We can see this by taking y = so 

1 — ax — j3x'^ — 70;^ = x^{y^ — ay^ — Py — ^) 

= x^{y -ri){y -r2){y -rs) 
= {1 — rix){l — r2x){l — rsx) 

What happens if we have a double root? I went over this in class, but I'm 
going to be lazy and not type it up. This case would make a good homework 
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problem when I teach the class again in the future. You probably did the case of 
a double root when you took calculus and learned the partial fractions method for 
integrating fractions with polynomials. If you remember what happened there, or if 
you remember what happens when you have a double root in differential equations, 
you can figure out what happens here. When you get r as a double root, you get a 
term with 

(an + /3)r'*, 

in the expression for /„, and if r appears as a fc-fold root, you get a term p[n)r'^ 
where p{n) is a polynomial of degree at most k — 1. 



3 A Chord Sequence 

Let's count something harder now. (This is the start of Monday's lecture). Let's 
count how many ways there are of putting chords in a fc-gon to divide it into 
triangles. We'll call this number Cfc_2. This sequence starts as follows: 

k 3 4 5 6 7 
j = k- 2 1 2 3 4 5 
Cj 1 2 5 14 42 

as you can see from the following figure. Here I've illustrated one of each essentially 
way of dividing a A;-gon into triangles, along with the number of times it must be 
counted (because of symmetry), for k <7. 




How can we find a recurrence for this number? Well, for a fc-gon, let's look at 
the triangle through the edge {k, 1). There must be a third point in this triangle. 
Let's call it j. Clearly, we must have 2 < j < k — 1. If we remove this triangle, we 
now have two smaller polygons, a j-sided one, and a {k — j + l)-sided one. We now 
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can divide these polygons up into triangles independently. (We need another figure 
here.) We thus get that the number of ways of triangulating a k-gon, given that we 
have a triangle with vertices k, is Cj-2Ck-j^i- 

One thing we notice is that for this to be true for j = 2 or j = A; — 1, we have 
to set Co = 1. This takes care of the case where j is 2 or A; — 1, and one of the two 
smaller polygons isn't really there (or is just an edge). 

Now, we can get a recurrence. Summing over all the j between 2 and k — 1 

k-l 

Ck-2 = Cj-2Ck-j-l 
3=2 

This formula can use some rethinking of the limits. Let's let fc' = k — 2 and f = j — 2. 
We get 

k'-i 

Ck' = ^ Cj>Ck'-j'-i 
j'=o 

which is a nicer-looking recurrence relation. 

The next question is how we evaluate it using generating functions. Let's look 
at the generating function for counting these triangulations That is, 

oo 

g{x) = CiXi = 1 + X + 2x^ + + 14x^ + ... 

i=0 

What happens when we square g{x). We get 

g{xf = l + (l + l)x + (l-2 + l-l + 2-l)x2 + (l-5 + 2-l + l-2 + l-5)x^ + ... 

k j=0 

You can see that the x^ term in the expression on the right is the right-hand-side 
of the recurrence relation we found above for (with = fc' — 1), so we get 

oo 

9{xf = Yl Ck+ix''. 
k=0 

Multiplying this by x gives a sum with the x^ coefficient equal to CjX^ . So we 
now have a expression relating xg{x)'^ and g{x). We need to make sure we get the 
smallest terms right. We can check the the constant term is the only one that's 
wrong, and we can fix that by adding 1 to xg{x)'^, to get the equation 

g{x) = 1 + xg(xf 
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Now, this is a quadratic equation in g{x), so we can use the quadratic formula to 
solve for it, and we obtain 

, , 1 ± VI - 4a; 
= 2-x • 

Well, we now have a choice here. Which of the two roots of this equation should we 
use? We can figure that out by looking at the first term. We should have g{0) = 1. 
Depending on which root we choose, we either get g(0) = 2/0 or g{0) = 0/0. Using 
I'Hopital's rule, or expanding the power series, we can figure out that in the second 
case, we indeed have ^(0) = 1, so we get 



, , 1 - VI - 4a; 
'^'^ = 2-x 

What is this number? Well, to figure that out, we have to expand g{x) in a power 
series. We can use the expansion 

n_y)l/2 _ , _ (l)(-l)(-3) , (!)(-!)( 3)(-5) _ (l)(-l)(~3)(~5)(-7) 

\^ y) ?2 2! 2^ ^3!^^ 2-* i\ 2^ 5! ^ ' " " 

Substituting y = 4x, this gives 

= '"^^ = E i^i^k - S)i2k - 5)i2k - 7) ... 1^ 

fc=i 

where by {2k — 3) {2k — 5) {2k — 7) ... 1 we mean the product of odd numbers between 
1 and 2k — 3, and 1 if A; = 1. All the — signs cancel out, as they should: since we're 
counting things, we have to get a positive integer. 

How do we simplify this expression? Let's equate coefficients on the left-hand- 
and right-hand-sides of this equation. The coefficient of x^ on the left-hand-side is 
Cj. On the right hand side, we have x'^"^, so we need to take k = j + 1. Equating 
coefficients, we get We get that 

_ 1 1 (2i-l)(2i-3)...l 

' 2 2i+i j + ll 

1 (2j-l)(2j-3)...l ^,. 
j + 1 j! 

Wc can now multiply the top and bottom of the above expression by j\ and simplify, 

to get 

1 {2j)l _ 1 (2j\ 

Here, we grouped the j! in the numerator with the 2^ and the product of odd 
numbers to get (2j)!. This is the j'th Catalan number (in class, I forgot to divide 
by X, and told you the wrong result, that it was the j — I'th Catalan number). 
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The Catalan numbers turn up in quite a few places. Prof. Richard Stanley has a 
section in his enumerative combinatorics book (this section is also on his web page) 
giving 66 combinatorial interpretations of the Catalan numbers. 



4 Ups and Downs 

The last example we will do is counting the number of alternating permutations. 
These are permutations that alternate up steps with down steps. For example, the 
permutation 3,6,1,8,4,7,2,5 is one, since 



Let An be the number of such permutations on the letters 1,2, ... ,n that start with 
an up step. It's an easy observation that if we know the number that start with 
an up step, we also know the number that start with a down step, because these 
numbers are equal by symmetry. If n > 1, it's also half the total number, since each 
alternating permutation (except when n = 1) either starts by going up or going 
down. 

How many such permutations are there? We'll go through this example a little 
more quickly than the last ones, since hopefully you are getting used to this kind 
of reasoning by now. We can easily count the first few terms of the sequence. This 
is a good idea in any case in working out this type of problem. You might notice 
a pattern in these numbers. Even if you don't, in working out the small cases you 
might get an idea of how to find a recurrence. And in any case, you can use these 
numbers to check your final results. We have 



We need to find a recurrence on the number of permutations. This means that we 
somehow need to build each such permutation out of smaller ones. How can we do 

this? 

It's not completely obvious how to find this recurrence. Here's how we do 
it. What we will do is remove the largest number in the permutation, which will 
break it into two pieces (one of these pieces may have length 0). Each of these 
pieces will alternate up and down steps, starting with up. And the first piece will 
always have odd length, since it has to start with an up step and end with a down 
step. In our example above, we remove the 8 and get the two sequences 3,6,1 and 
4,7,2,5. These aren't examples of alternating permutations, since they aren't on the 
mimbers 1,2, ... ,k for any k. However, we can map each of these to a permutation 
on 1,2, ... ,k while preserving the relative order of these numbers. For our example 



3<6>1<8>4<7>2<5 



A; 1 2 3 
^fe 1 1 1 2 



4 5 

5 16 
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above, these smaller permutations (counted ^3 and A4 respectively) would be 2,3,1 
and 2,4,1,3. 

Now, however, we've lost some information about how to construct the original 
permutation. We need to know which numbers were on the right and which were 
on the left. That is, we need to know which subset ofl,2,3,...,n — 1 was to the 
left of n. In our example above, this subset was {1,3,6}. If we know this subset, 
we can go back and figure out what our original sequence was. 

Furthermore, we have found a one-to-one correspondence between, on the one 
hand, alternating permutations of size n, and, on the other, a subset of some odd 
size A;of{l,2,...,n — 1} together with a pair of alternating permutations of size k 
and n — k — 1. This is because if we start with an subset of odd size, and such a 
pair of alternating permutations, the reconstruction will always give us a length n 
alternating permutation. This gives us the recurrence 



since ("^^) is the number of subsets of size k. 

We can get almost the same recurrence with even k, by removing the smallest 
element, 1, instead of the largest. The second alternating permutation (the one to 
the right of the 1) now starts going down instead of up, but this doesn't matter for 
our recurrence, as the same number start up and down. This gives the recurrence 

2L{n-l)/2j _ 




k=0 
k even 



k 

Adding these two equations together, we get the simpler recurrence 

= J2(^ J. ^^AkAn-k-l- 



We now need to find a generating function. Our generating function will be 



00 j 

X-' 



5(x) = ^A,-. 
j=o ■'■ 

Note that we have stuck in a j\ which we didn't in our previous examples. This is 
a pretty common thing to do in making generating functions. If you do it, what 
you get is called an "exponential generating function," and if you don't do it, the 
technical term for what you get is an "ordinary generating function." 
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Why did we choose an exponential generating function here? The first reason 
is the this j\ ensures that the generating function converges, whereas if we tried to 
write down an ordinary generating function, the function Aj grows too fast for it 
to converge. The second reason is that the jl combines with the ("y^^) term in the 
recurrence very nicely so that they both nearly cancel. We'll see that now. 

If we let Bj = Aj/j\, our exponential generating function for Aj becomes an 
ordinary generating function for Bj. We have, expanding the term ("^ ) in our 
recurrence, 

n-1 

ifc ^n-k-l 

n - IJ!- 

fe=0 

Now replacing Aj with Bj above, we get 



or simplifying. 



n-1 

2n!5„ = 5^(n-l)!SfeS„_jfc_i, (1) 
fe=o 

n-1 

2nBn = ^ BkBn-k-i- (2) 

A;=0 

We know how to handle the right side of this equation using generating functions. 
We saw this expression when we were looking at the Catalan numbers. This is just 
the x"'~^ term of g{x)^. How do we get the left side? Let's remember that 

9ix) = ^BjxK 

3 

Differentiating, we get 

g'{x) = Y.jBjX^-\ 

3 

Thus, the left side of (4) above is the term of ^'(a;). So we now have an 

equation 

2g'{x) = g{xf + l 

where I've left a '?' in the equation we haven't chekced that the coefficients for 
small powers of x are equal in this equation, so we may have to add some small 
polynomial in x to fix any problems. In fact, this equation doesn't work for the 
constant term. This is because our recurrence doesn't work for Aq (this is easy to 
check) . 

It's easy to calculate B]^. The sequence is 



k 



1 2 3 4 5 

-1 -I 1 1 _5_ _2_ 
^ 2 3 24 15 
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and from these numbers, we can substitute into the differential equation and figure 
out what the "?" has to be to get it to work. We find 



2g'{x)=g{xf + 1 

What's the solution to this? I don't know how to figure out the solution to this 
differential equation by hand, but you can always type it into Maple or Mathematica. 
If you use Maple, it tells you that the solutions are 

g{x) = tan(| + 9) 

for some constant 9, and it's easy to check that this works by differentiating. We 
need to choose 9 so that ^(0) = 1. It's easy to see that this means 9 = f , so we get 

/ \ f X tt\ 

g{x) = tan + -J . 

We can use trigonometric identitites to get the equivalent expression 

g{x) = tan x + sec x 

which is a more useful expression for looking at Af^. This is because tan is an odd 
function, while sec is an even function, so the odd Ak come from the expansion of 
tanx while the even ones come from the expansion of secx. These are well-studied 
power series, and the odd A;, are called tangent numbers while the even A^ are 
called secant numbers. 

The generating function for g{x) can tell us quite a bit about the asymptotic 
growth rate of the numbers A^, and I'll get to putting that in these notes later. 
That stuff isn't urgent, since it won't help you with the homework. 
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